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This article examines the psychometric properties of the 1 04-item Thinking Styles 
Inventory (TSI; Sternberg & Wagner, 1992) using responses from 789 students 
from 4 high schools in Connecticut. Twelve of the 13 subscales identified in mental 
self-government (MSG) theory (Sternberg, 1988, 1997) were included in all analy- 
ses. Both subscale- and item-level confirmatory factor analysis failed to confirm the 
theory-proposed 5-factor structure as well as 3 other structural models identified in 
previous studies. Post hoc item-level exploratory factory analysis and subscale score reli- 
ability analysis resulted in the omission of 64 original items due to weak psychometric 
properties. Thirty-two original items were retained, comprising 5 subscales, Liberal/ 
Progressive, External, Hierarchic, Judicial, and Legislative/Seif-Reliant, largely con- 
sistent with those identified in MSG theory. Implications of the results are discussed. 




Although by no means a new area of research in the field of education 
or psychology, the study of intellectual (cognitive, thinking, learn- 
ing) styles continues to provide new information about their role in 
students’ academic experiences that has implications for educational 
practice. For more than 5 decades, theorists have named and defined 
numerous categories of intellectual styles, a fairly comprehensive 
account Sternberg and Grigorenko (1997, 2001) provide. Over the 
past decade, one theory that has received considerable attention in 
educational research has been Sternberg’s (1988, 1997) theory of 
mental self-government (MSG), which proposes that the organiza- 
tion of thinking parallels the organization of political government, 
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and that individuals govern themselves according to their personal 
thinking styles or, more accurately, their own profiles of thinking 
styles. Thinking style has been defined as the “interface between 
intelligence and personality” (Sternberg, 1994, p. 169), and as . a 
favored way of expressing or using one or more abilities” (Grigorenko 
& Sternberg, 1997, p. 297). The construct has been operationalized 
by the Thinking Styles Inventory (TSI; Sternberg & Wagner, 1992), 
a 104-item survey that assesses individuals’ preferences on each of 
13 styles. This article examines the psychometric properties of that 
instrument in order to determine the accuracy with which the con- 
struct MSG thinking styles has been measured and thus the extent to 
which research related to that construct should be regarded as well- 
founded. 

The term style, as regards human intellect, refers to “habitual pat- 
terns or preferred ways of doing something . . . that are consistent over 
long periods of time and across many areas of activity” (Sternberg & 
Grigorenko, 2001, p. 2). Style is differentiated from ability, a term 
referring to “things one can do — a skill or skill combinations” ( Zhang, 
2001, p. 622). MSG theory proposes that thinking styles comprise 
five dimensions that are analogous to facets of government: function, 
form, level, scope, and leaning (Sternberg, 1997; see Table 1). 


MSG Dimensions of Thinking Style 

The three functions of mental self-government are legislative, execu- 
tive, and judicial. Briefly, legislative thinkers enjoy creating their own 
rules for doing things, and prefer to decide for themselves what things 
to do and how to do them. Executive thinkers, conversely, prefer to 
follow established rules and value problems that are prestructured. 
People with a preference for a judicial style of thinking favor analyz- 
ing and evaluating existing rules and procedures and critiquing the 
work of others (Sternberg, 1997). 

There are four forms of mental self-government: monarchic, 
hierarchic, oligarchic, and anarchic (Sternberg, 1997). The monar- 
chic person is described as “single minded and driven” (Sternberg, 
1997, p. 22), focusing exclusively on tasks or activities of inter- 
est. The hierarchic person is able to prioritize among goals and 
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“recognizes the need to view problems from a number of angles” 
(Sternberg, 1997, p. 23) as a means of setting priorities. To be oli- 
garchic in thinking is to have the tendency to be driven simultane- 
ously by several goals of seemingly equal importance with difficulty 
managing the conflicting demands. Lastly, the anarchic individual 
seems to take a random approach to problems, resisting the con- 
finement of rigid systems. 

In terms of levels , global thinkers prefer to deal with large and 
abstract issues rather than details while local thinkers enjoy working 
with concrete problems and tend to orient toward details (Sternberg, 
1997). 

Having an internal scope of mental self-government involves 
focusing on internal matters. Individuals who favor an internal style 
are more introverted, task oriented, and perhaps less socially sensitive 
than are external people. While internal-style individuals prefer to 
work alone, externals enjoy working with other people and tend to 
be outgoing (Sternberg, 1997). 

Leanings refer to individual tendencies to embrace or eschew 
change. Liberal individuals seek opportunities to “go beyond exist- 
ing rules” (Sternberg, 1997, p. 26) to create change, and may become 
bored easily. Conservatives tend to prefer to follow existing rules 
and avoid unfamiliar situations. Individuals favoring this thinking 
style will create structure in environments where structure is lacking 
(Sternberg, 1997). 

MSG theory asserts that people possess profiles of styles rather 
than single styles (Sternberg, 1997). A profile is composed of some 
combination of the elemental styles within the five MSG dimen- 
sions. A thinking style profile is made up of several favored styles. For 
example, a student may favor both legislative and internal styles, indi- 
cating a preference both to rely on her own ideas and strategies for 
solving a problem and to work independently. Further, and perhaps 
most intriguing, is the proposition that thinking styles are socialized 
and teachable, variable according to task and situation, and, to some 
degree, flexible (Sternberg, 1997). Although an attractive notion, this 
seems to conflict with the very nature of styles as being “habitual pat- 
terns . . . that are consistent over long periods of time . . .” (Sternberg 
& Grigorenko, 2001, p. 2). Nonetheless, some degree of empirical 
support for the socialization of styles has been claimed. 
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Social and Academic Correlates of MSG Thinking Styles 

Sternberg and Grigorenko (1995) used correlational analysis to 
illustrate “at least a suggestion” that students’ thinking styles may be 
socialized by their teachers (p. 213) and reported significant correla- 
tions between some thinking styles and socioeconomic status (SES), 
as estimated by parent education (viz., executive style, r = -.43; with 
judicial style, r = -.23; and with conservative style, r = -.47; p. 212) 
and as estimated by father education only (viz., legislative style, r = .36 
and with hierarchic style, r = .25) . Additional support was declared 
by Zhang and Postiglione (2001) who, after controlling for student 
age, found weak but statistically significant correlations between 
some thinking styles and SES indicators. Those partial correlations 
ranged from r = .08 (global style and fathers education level) to r = 
.12 (hierarchic style and family income). In addition to considering 
the size of the reported correlation coefficients, the critical reader 
also should note that causal relationships were not established in 
these studies. That is, neither differences in teachers’ thinking styles 
nor differences in students’ family SES was determined to cause dif- 
ferences in students’ thinking styles. 

Other studies have reported associations between thinking style 
and academic achievement (Bernardo, Zhang, & Callueng, 2002; 
Cano-Garcia & Hewitt Hughes, 2000; Zhang, 2000, 2002a) as well 
as interactions between thinking styles and types of academic assess- 
ment (Sternberg & Grigorenko, 1993), between students’ and teach- 
ers’ thinking styles (Sternberg & Grigorenko, 1995), and between 
thinking styles and academic subject (Zhang, 2001, 2004) that have 
implications for students’ academic performance. Some studies have 
called upon educators to revise instructional and assessment strate- 
gies to accommodate students of varying thinking styles (Sternberg 
& Grigorenko, 1993, 1995; Zhang, 2000). 

The measurement and application of thinking styles have been 
advocated within the gifted education literature (Dai & Feldhusen, 
1999; Sternberg & Grigorenko, 1993). Sternberg and Grigorenko 
suggested that thinking styles are likely moderator variables that may 
help to distinguish different subgroups of gifted children. They main- 
tain that thinking styles may explain preferences for independence or 
guided instruction and individual work or cooperative group work. 
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Further, they suggest that thinking styles may inform the nature of 
assessment used for identification of giftedness, and thinking style 
should be considered when making decisions between educational 
enrichment and acceleration options (Sternberg & Grigorenko, 
2003). However intuitively appealing these recommendations may 
be, primary consideration must be given to the reliability and valid- 
ity of the measurement of the thinking styles construct. The savvy 
consumer of any such research should first ask, “How well has this 
construct been measured?” 




Measurement of MSG Thinking Styles 

The 13 thinking styles that compose the five dimensions proposed in 
MSG theory are operationalized by the TSI (Sternberg & Wagner, 
1992), a 104-item scale with eight questions targeting each style. 
Individuals rate the eight items within each subscale from 1 -7 indi- 
cating how well each statement describes them, where 1 = not at all 
well and 7 = extremely well. A mean subscale rating that is close to 7 is 
a high score and is interpreted as a preference for that thinking style. 
Mean ratings closer to 1 are interpreted as “low” ratings for that sub- 
scale and indicate a lack ofpreference for that style (Sternberg, 1997). 
Thinking style profiles are identified by the TSI in terms of how high 
or low respondents’ mean scores are on each of the 13 thinking style 
subscales. A profile is the combination of all subscales on which the 
rater indicates a preference. 

Variations of the TSI include an expanded item rating scale (1-9 
vs. 1-7) and a shortened, 65-item version (General Thinking Styles 
Inventory-Short Version, unpublished) consisting of 5 items for 
each of the 13 subscales. 

Although Sternberg (1997) claimed that the TSI has “demon- 
strated good psychometric properties” (p. 125), studies using scores 
on the TSI have returned varying results in terms of subscale score 
reliability (coefficient alpha). All have indicated a wide range of esti- 
mates across the 13 subscales (see Table 2). With the full 104-item 
scale for four subgroups of students divided by grade (grades 7 & 8, 
9 & 10, 1 1 & 12, and college) and one group of “laypeople,” subscale 
score reliabilities ranged from .35 to .88 across groups, with mean reli- 
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Note. Values above the bolded line reflect estimates for the full-scale instrument. Values below the bolded line reflect estimates from the reduced, 65-item instru- 
ment. a Subjects were in grades 7 & 8; b subjects were in grades 9 & 10; c subjects were in grades 1 1 & 12; d subjects were college students; e subjects were laypeople; 
subjects were from a Hong Kong sample; g subjects were from a Nanjing sample; h subjects were from a U.S. sample. 
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ability estimates ranging from .63 (grades 7 & 8) to .76 (laypeople; 
Sternberg & Wagner, 1992). Other reported alpha reliability ranges 
for scores obtained with the full-scale instrument were .38-. 86 (Dai 
& Feldhusen, 1999) and .35-. 88 (Cano-Garcia & Hewitt Hughes, 
2000). Studies using the 65-item inventory have reported similar 
alpha reliability ranges of .46-. 82 (Zhang, 2004); .51-. 86 (Zhang 
2002c); .44-. 80, .13-.81, .50-.86 (Zhang, 2000); and .50-.81 
(Bernardo et ah, 2002). Although there is no universally accepted 
minimum estimate for the internal consistency coefficient, Charter 
(2003) presented many proposed guidelines in a review of literature 
on the topic. Minimum alpha values of .80 to .90 are the most com- 
mon among those recommendations. According to Charter, “The 
ideal of an internal consistency reliability of .90 or .95 may be dif- 
ficult to attain in many cases, but test constructors should strive to 
give clinicians good tools to work with” (p. 302). 

In addition, the factor structure of the subscales of the TSI has not 
been clearly established, with factor analysis failing to fully support the 
dimensionality proposed by MSG theory (Sternberg & Grigorenko, 
1993) and with other factor analytic studies reporting inconsistent fac- 
tor structures across samples (see, for example, Bernardo et ah, 2002; 
Cano-Garcia & Hewitt Hughes, 2000; Dai & Feldhusen, 1999; Fjell 
& Walhovd, 2004; Zhang & Sachs, 1997; see Table 3). “If the factor 
structure of a measure cannot be replicated from sample to sample, 
its usefulness is diminished, perhaps to the point where its construct 
validity is called into question” (Steger, 2006, p. 263). 

Moreover, we have found no reported item-level factor analysis on 
the full-scale TSI. Although a confirmatory factor analysis (CFA) of 
the 24 items composing the three MSG Functions was conducted by 
O’Hara and Sternberg (2000-2001; results of which indicated poor 
model fit; p( 2 (255, N = 105) = 548, p < .001; goodness of fit index = 
.71), our review of the literature revealed that all factor analyses con- 
ducted on the full-scale instrument involved subscale -level data. 




Rationale for Study 

The current study is unique in its goal to identify the factor struc- 
ture of TSI scores for high school aged students using item-level data. 


JEG 32(2) Winter 2008.indd 1 88 




12/12/08 8:55:52 AM 


T5I Validity Study 


189 


Table 3 

Factor Structures of theTSI Reported in Six Studies 


Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 


Initial Study 

Conservative 

Judicial 

External 

Local Hierarchic 

(see Sternberg 

Executive 

Oligarchic 

Internal (-) 

Global (-) 

& Grigorenko, 

Progressive (-) 




1993) 

Legislative (-) 




Zhang & Sachs, 

Conservative 

Legislative 

JudiciaP 


1997 

Executive 

JudiciaP 

Liberal 1 



Oligarchic 

Local 

Hierarchical 



Monarchic 

Liberal 1 

Anarchic 

Internal 

External 


Dai & 

Legislative 

Executive 

Internal (-) 1 


Feldhusen, 1999 

Liberal 

Conservative 

External 



Anarchic 

Monarchic 

Global 



Judicial 

InternaP 

Local 

Hierarchic 



Cano- Garcia & 

Progressive 

Executive 

Internal 

Monarchic 1 

Hewitt Hughes, 

Judicial 

Conservative 1 

External (-) 1 

Global 

2000 

Anarchic 

Hierarchic 

Legislative 1 

Local (-) 


Conservative t 

Monarchic 1 

Oligarchic (-) 1 



ExternaP 

Legislative 1 

Oligarchic 1 



Bernardo, 

Legislative 

Executive 

Oligarchic 1 


Zhang, & 

JudiciaP 

JudiciaP 

Internal (-) 1 


Callueng, 2002 

Global 

Local 

External 



Liberal 

Conservative 




Anarchic 

Hierarchic 




InternaP 

Monarchic 

Oligarchic 1 



Fjell & 

Judicial 

Legislative 1 

Anarchic 

External Local 

Walhovd, 2004 

Hierarchic 

Executive 

Oligarchic 

Internal Global 

Norwegian 

Liberal 

Conservative 



Sample 

Legislative 1 

Monarchic 



Fjell & 

Judicial 

Executive 

Hierarchic 1 

External Global 

Walhovd, 2004 

Hierarchic 1 

Conservative 

Anarchic 

Internal 

Texas Sample 

Liberal 

Monarchic 

Oligarchic 



Legislative 

Local 




Note. Subscales with coefficients > .40 are listed when provided. Otherwise, the factor structure 
is reproduced as reported in the article. + Subscale loaded on more than one factor. ( ) Negative 
factor loading. 
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The study used CFA to compare the adequacy and fit of competing 
hypothesized structural models and used posthoc exploratory factor 
analysis (EFA) to examine further the data structure. A second pur- 
pose of the study was to determine whether the TSI could be reduced 
to a shorter, psychometrically stronger inventory that provides a reli- 
able measure of MSG thinking styles. 


Methods 

In this study, scores on the 104-item TSI were collected from 790 
students in grades 9-12 in four high schools in Connecticut. One 
case was removed due to substantial missing data, resulting in 789 
cases. For the remaining cases, the amount of missing data ranged 
from .25% to 3.80% per variable. Four-hundred seventy-seven cases 
contained complete data and 222 different patterns of missingness 
existed for the remaining cases. There was no reason to believe data 
were missing by any systematic process, and we assumed the data to 
be missing at random (MAR), an assumption that is, in many cases 
“quite plausible” (Schafer & Graham, 2002, p. 152). In order to 
retain all cases for analysis, data augmentation with single imputation 
of estimates for missing values 1 was used with NORM 2.03 software 
(Schafer, 2000). This resulted in 789 cases with completed (observed 
and simulated) data for 12 of the 13 subscales (96 of the 104 total 
items). Because different items were inadvertently used to measure 
the oligarchic style with some of the students (see Oligarchic items in 
Sternberg & Wagner, 1992, vs. Sternberg, 1997), that 8-item subscale 
was omitted from all analyses. 

TSI subscale score reliabilities were estimated with SPSS 14.0 
software and compared to previous studies. We then conducted 
CFAs of both scale- and item-level data with AMOS 6.0 to deter- 
mine whether the MSG theory-specified model structure (Sternberg, 
1988, 1997) fit the data collected from this high school sample. 
Three other proposed structures also were specified using both sub- 
scale- and item-level data for comparison of model fit; one model 
specifying the threefold model of intellectual styles with three factors 
indicated by Type I, Type II, and Type III styles (Zhang 2006; Zhang 
& Sternberg, 2005); the second specifying two factors made up of 
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Type I and Type II styles (Zhang, 2002a, 2002b), and the third speci- 
fying the four-factor structure reported by Zhang (2006). Item-level 
EFA was subsequently conducted with SPSS 14.0 to examine further 
the latent structure accounting for the covariances among variables 
in the data set (Brown, 2006). Finally, Cronbach’s alpha internal con- 
sistency reliabilities were estimated for the subscales composed of 
retained items. 




Data Screening 

As an initial screening of the data, the distributions of items were 
examined for univariate normality. In addition, bivariate and squared 
multiple correlations among the 12 subscales were reviewed as indica- 
tors of multicollinearity, and variances of subscales were examined for 
relative size. Subscale score internal consistency reliability estimates 
also were assessed (Kline, 2005). All items were distributed normally 
(absolute value of skewness index < 3.0, absolute value of kurtosis 
index < 10.0; Kline, 2005). Absolute values of the bivariate correla- 
tions among subscales ranged from r — .02 (Fiberal-Conservative) 
to r = .65 (Fiberal-Fegislative), and no squared multiple correlation 
was larger than R 2 = .61 , signifying nonredundancy among subscales. 
Subscale variances were homogeneous, ranging from .74 to 1.27. 
However, reliability estimates for some subscale scores (Monarchic, 
Anarchic, and Focal) were lower than .70, a generally recommended 
value for “adequate” reliability (Kline, 2005). 


Results 


Internal Consistency Reliability Analysis 

Internal consistency reliabilities for scores on the original 12 TSI 
subscales were estimated for comparison to previous studies. Our 
estimates were higher in comparison, ranging from .64 (Monarchic) 
to .84 (Fiberal). However, in addition to reporting Cronbach’s 
alpha, which is solely a function of the interrelatedness of items 
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Table 4 

Internal Consistency Reliabilities, Standard Errors, 
and Interitem Correlations for 12 
of the Original 1 3 Subscales of the TSI 


Subscale 

Coefficient alpha 
(SE) 

Mean IIC 

Minimum 

IIC 

Maximum 

IIC 

Std Dev of 
IIC 

Anarchic 

.67 (.014) 

.204 

.003 

.336 

.071 

Conservative 

.81 (.020) 

.350 

.142 

.571 

.105 

Executive 

.80 (.016) 

.331 

.141 

.534 

.084 

External 

.82 (.024) 

.364 

.119 

.617 

.126 

Global 

.71 (.025) 

.244 

-.042 

.465 

.130 

Hierarchic 

.80 (.012) 

.330 

.218 

.472 

.063 

Internal 

.79 (.019) 

.320 

.131 

.571 

.100 

Judicial 

.73 (.023) 

.252 

.021 

.441 

.118 

Legislative 

.81 (.015) 

.340 

.177 

.512 

.077 

Liberal 

.84 (.019) 

.392 

.218 

.637 

.100 

Local 

.67 (.011) 

.206 

.107 

.388 

.055 

Monarchic 

.64 (.017) 

.185 

.063 

.485 

.089 


Note. IIC = Interitem correlations, SE = standard error. 


in the subscale and test length and not of scale unidimensionality 
(Schmitt, 1996), it is recommended that the “precision of alpha” also 
be reported (Cortina, 1993, p. 100), providing an indication of the 
spread of interitem correlations (IIC). Both precision of alpha esti- 
mates and standard deviations of IIC were calculated for all subscales 
(see Table 4). 

A large spread of correlations “indicates either some form of mul- 
tidimensionality or a great deal of sampling error in the estimation of 
the interitem correlations” (Schmitt, 1996, p. 351). 

Anarchic, Local, and Monarchic subscales had particularly 
weak mean IIC (.20, .21, and .19, respectively) while Conservative, 
External, Global, Internal, Judicial, and Liberal subscales had large 
IIC ranges with relatively unstable alpha estimates (SD nc > .100; 
SE.^ - 0.019). These statistics are indicative of errors in the opera- 
tionalization of the proposed constructs that also were evident in the 
factor analyses. 
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Subscale-Level CFA 

The first subscale-level CFA tested the five-factor model specified 
by MSG theory (Sternberg, 1988, 1997) with forms, function, level, 
scope, and leaning identified as latent variables causing the vari- 
ance in scores on each of their respective style indicators (omitting 
oligarchic as an indicator variable of the Function factor). Factor 
1, Forms, was indicated by Legislative, Executive, and Judicial sub- 
scales. Factor 2, Function, was indicated by Anarchic, Monarchic, 
and Hierarchic subscales. Factor 3, Level, was indicated by Global 
and Local subscales. Lactor 4, Scope, was indicated by Internal and 
External subscales. Lactor 5, Leaning, was indicated by Liberal and 
Conservative subscales. Correlations between all factor pairs were 
specified. Because a sample variance was estimated to be zero or nega- 
tive, no model solution could be estimated. Negative variance esti- 
mates are indicative of some form of model misspecification (Kline, 
2005). To test a more parsimonious model, an equality constraint 
was then applied to the respective paths of the three factors with only 
two indicator variables. This respecification did not improve model 
fit, and no solution could be estimated for this revised model. One 
source of misspecification of the five-factor model was the specifica- 
tion of weakly correlated pairs of indicators for factor 4 (Internal 
and External; r = -.11) and Lactor 5 (Liberal and Conservative; r = 
.02), with the bivariate correlation for the latter pair not statistically 
significantly different from zero (see Table 5). 

Next the three-factor Type model (Zhang & Sternberg, 2005) 
was fit to the data, specifying Legislative, Judicial, Hierarchic, 
Global, and Liberal styles as indicators of Type I; Executive, Local, 
Monarchic, and Conservative styles as indicators of Type II; and 
Anarchic, Internal, and External styles as indicators of Type III. The 
result was an inadmissible solution, with the correlation between 
Type I and Type III factors estimated to be greater than 1.0 and the 
variance-covariance matrix for the first-order factors not positive- 
definite. 

Then we estimated the fit of the two-factor Type model (Zhang, 
2002a, 2002b) with Lactor 1, Type I, causing Legislative, Judicial, 
Hierarchic, Global, and Liberal variables and factor 2, Type II, caus- 
ing Executive, Local, Monarchic, and Conservative variables. The 
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result indicated poor fit of the model to the data (p( 2 [26] = 653.29; 
p < .001; CFI = .779; RMSEA = .175 [90% C.I. = .164, .187]; 
PCLOSE < .001). Large modification indices suggested correlat- 
ing several uniquenesses with latent variables, correlating uniqueness 
pairs, and double-loading subscale indicators on both factors. 

Finally we tested the four-factor model (Zhang, 2006) with 
Factor 1, Type 1, indicated by Legislative, Liberal, and Judicial sub- 
scales; Factor 2, Type II, indicated by Executive, Conservative, and 
Monarchic subscales; Factor 3, Level, indicated by Global and Local 
subscales, and Factor 4, Scope, indicated by Internal and External 
subscales. For this model, as with the five-factor model, a sample 
variance was estimated to be zero or negative, and no model solution 
could be estimated. 

Item-Level CFA 

The same four models were then estimated with item-level data. These 
models were hierarchical, with Dimension or Type factors serving as 
second-order factors and subscale factors as first-order factors, each 
indicated by their respective eight items (see Sternberg & Wagner, 
1992). In each model, all second-order factor pairs were specified to 
be correlated. Disturbances were added to all first-order factors and 
their respective paths were set to 1.0. 

The hierarchical five-factor model (Sternberg, 1988) was esti- 
mated first. In this model, there were 5 second-order factors, Forms, 
Function, Level, Scope, and Leaning. Forms was indicated by 3 
first-order factors, Legislative, Executive, and Judicial. Function 
also was indicated by 3 first-order factors, Anarchic, Monarchic, 
and Hierarchic. Level was indicated by 2 first-order factors, Global 
and Local. Scope was indicated by 2 first-order factors, Internal and 
External. Finally, Leaning was indicated by 2 first-order factors, 
Liberal and Conservative. All first-order factors were indicated by 
the eight measured variables that correspond with those subscales in 
the TSI. The result of this CFA was the same as with subscale-level 
data; a sample variance was estimated to be zero or negative, and no 
model solution could be estimated. 

Next, the hierarchical three-factor Type model (Zhang, 2006; 
Zhang & Sternberg, 2005) was specified. In this model, Type I, Type 
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II, and Type III were specified as second-order factors. The first- 
order factors indicating Type I were Legislative, Judicial, Hierarchic, 
Global, and Liberal. Indicating Type II were first-order factors 
Executive, Local, Monarchic, and Conservative. Anarchic, Internal, 
and External first-order factors were indicators of the second-order 
Type III factor. All first-order factors were indicated by the eight 
observed variables that correspond with those subscales in the TSI. 
The result of this CFA was the same as with the subscale-level data: 
The solution was inadmissible, with the correlation between Type I 
and Type III factors estimated to be greater than 1 .0 and the variance- 
covariance matrix for the first-order factors not positive-definite. 

The hierarchical two-factor Type model (Zhang, 2002a, 2002b) 
was then specified. In this model, there were 2 second-order factors, 
Type I and Type II. The former was indicated by 5 first-order fac- 
tors, Legislative, Judicial, Hierarchic, Global, and Liberal, whereas 
the latter was indicated by 4 first-order factors, Executive, Local, 
Monarchic, and Conservative. All first-order factors were indi- 
cated by the eight observed variables that correspond with those 
subscales in the TSI. The result indicated poor fit of the model to 
the data (x 2 [2474] = 8000.128; p < .001; CFI = .696; RMSEA = 
.053 [90% C.I. = .052, .055]; PCLOSE < .001). The standardized 
regression weight and model-implied correlation between Type II 
and Monarchic was .994, suggesting redundancy between those sec- 
ond- and first-order factors. Large modification indices suggested 
correlating several disturbances with first-order factors, correlating 
uniquenesses with latent variables, and double-loading indicator 
variables. 

Finally, the four- factor model (Zhang, 2006) was specified with 
item-level indicators. This model had 4 second-order factors, Type I, 
indicated by Legislative, Liberal, and Judicial first-order factors; Type 
II, indicated by Executive, Conservative, and Monarchic first-order 
factors; Level, indicated by Global and Local first-order factors; and 
Scope, indicated by Internal and External first-order factors. All first- 
order factors were indicated by the eight observed variables that cor- 
respond with those subscales in the TSI. The result was the same as 
with subscale-level data; a sample variance was estimated to be zero 
or negative, and no model solution could be estimated. 
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Item-Level EFA 

Subsequent to CFA in which a model is “grossly misspecified,” EFA 
may be conducted to reassess the structure of the measurement 
model (Brown, 2006, p. 189). Given the failure of CFA to confirm 
any of the four proposed models with either scale- or item-level data, 
an item-level EFA of 12 of the 13 subscales (omitting Oligarchic) 
in the full-scale TSI was then conducted using SPSS 14.0 software. 

Because some subscale pairs have been determined to correlate sig- 
nificantly (see Sternberg & Grigorenko, 1993), we used the principal 
axis factoring extraction option, selecting the extraction of factors 
with eigenvalues > 1.0, with direct oblimin rotation. Scree plot, par- 
allel analysis, amount of variance explained by each factor, and con- 
sistency with the theoretical model informed the extraction decision 
(Kim & Mueller, 1978; Thompson, 2004). Scree plot visual analysis 
(Cattell, 1978) indicated the extraction of four factors, and parallel 
analysis indicated the extraction of nine factors. The first 4 factors 
accounted for only 31% of the total variance, whereas nine factors 
accounted for 43% of the variance. Given the greater percentage of * 

variance accounted for with a nine-factor solution and the closer 
agreement of this solution with Sternberg’s proposed model of 13 
factors, a nine-factor solution was specified for extraction, and the 
patterns of coefficients resulting from that solution were analyzed. 

In addition to item content, weak pattern coefficients on primary 
factors (< .350) and/or coefficients on secondary and subsequent 
factors of magnitude similar to the primary coefficient informed the 
decision to retain items. Forty-four of the 96 items, subsumed by six 
factors, met the criteria for retention (see Table 6). 

Internal consistency reliability analysis of subscale scores was 
then conducted. As coefficient alpha is the quantification of the 
“degree of interrelatedness among a set of items designed to mea- 
sure a single construct” (Netemeyer, Bearden, & Sharma, 2003), 
items weakly correlated with other items in the subscale were deter- 
mined to be poorly related to the construct and were removed. This 
process resulted in the omission of 12 additional items. Ultimately, 
all items from the Executive, Monarchic, Anarchic, Global, Local, 
and Conservative subscales were eliminated. Seven items from the 
Internal subscale, three items from the Judicial subscale, two items 
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791 like to challenge old ideas or ways of doing things and to seek better ones. -.535 

25 I like projects that allow me to look at a situation from a new perspective. -.489 

211 like to play with my ideas and see how far they go. -.479 .274 
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77 I like situations where I can focus on general issues, rather than on specifics. .318 .490 

111 like situations or tasks in which I am not concerned with details. .483 
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4 7 1 use any means to reach my goal. .358 

88 If there are several important things to do, I do the one most important to me. .292 .344 

2 1 prefer to deal with specific problems, rather than with general questions. .342 -.205 
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each from the Hierarchic and External subscales, and one item from 
both the Legislative and Liberal subscales were removed. This pro- 
cess resulted in the retention of 32 items subsumed by five factors 
with internal consistency reliabilities ranging from .729 to .863 (see 
Tables 7 and 8). Mean IIC within subscales were moderate, rang- 
ing from .35 (Judicial) to .45 (External), with standard deviations 
< . 10 . 

Generally, items within the remaining original subscales loaded 
together on a common factor, with some items from related subscales 
also loading with those factors. The first subscale, for example, con- 
tains seven of the original Liberal items and two Legislative items. 
(The correlation between those two original subscales was reported 
to be .66. See Sternberg & Grigorenko, 1993). The items in the 
new subscale combine to measure a Liberal/Progressive approach 
to thinking, with a preference for trying new methods and finding 
new strategies to solve problems. The second factor contains seven 
items exclusively from the original External subscale, measuring indi- 
viduals’ preferences for working and sharing ideas with others as out- 
lined in MSG theory, factor 3 subsumes six items from the original 
Hierarchic subscale. This factor measures the degree to which indi- 
viduals prefer to order ideas and things to do by perceived impor- 
tance. factor 4 contains five Judicial items describing a propensity 
toward comparing and rating ideas or views, finally, factor 5 con- 
tains five Legislative and one Internal item, combining to measure 
individuals’ reliance on their own ideas and strategies when doing a 
task (Legislative/Self-Reliant). Correlations between subscales were 
weak to moderate and positive with the strongest correlation {r = 
.56) between Liberal and Legislative subscales and the weakest (r = 
.16) between Hierarchic and Judicial subscales (see Table 9). 




Discussion 

Our study failed to provide statistical support for the use of the 
full-scale TSI as an instrument for the operationalization of MSG 
thinking styles with high school students. Although internal consis- 
tency reliability analysis of subscales returned alpha estimates that 
were larger than previous reports, interitem correlations within some 
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Table 8 

Internal Consistency Reliabilities of the Five Factors 
Subsuming the Retained 32 Items of the TSI 



Number 

Coefficient 


Minimum 

Maximum 

Std Dev of 

Subscale 

of Items 

alpha (SE) 

Mean IIC 

IIC 

IIC 

IIC 

Liberal/ 

Progressive 

9 

.863(.013) 

.412 

.305 

.638 

.077 

External 

6 

.832(.027) 

.448 

.285 

.617 

.100 

Hierarchic 

6 

.767(.015) 

.355 

.278 

.472 

.055 

Judicial 

5 

.729(.024) 

.351 

.249 

.439 

.071 

Legislative/ 

Self-Reliant 

6 

.778(.021) 

.368 

.243 

.512 

.077 


Note. SE = Standard Error, IIC = Interitem correlations. 


subscales were weak and varied, suggesting substantial measurement 
error. Neither scale- nor item-level CFA confirmed any of four pur- 
ported latent structures of scores, revealing poor model fit or inad- 
missible solutions for the specified models. While a combination of 
item-level EFA and reliability analysis led to the identification of 32 
relatively strong items that loaded on factors similar to those pro- 
posed by MSG theory, the scope of the remaining items does not suf- 
ficiently account for the theorized range of thinking styles as outlined 
by Sternberg (1988, 1997), nor does it provide a readily interpreta- 
ble reduction in proposed styles. What may be considered a useful 
result of the EFA is the identification of items that are psychometri- 
cally strong, which may provide the basis for the development of an 
improved inventory of thinking styles. The study suggests that some 
items from the Liberal and Legislative, and Legislative and Internal 
subscales are conceptually similar and share common variance, indi- 
cating common latent structure among items in those subscales. This 
is consistent with ELA results from previous studies (Bernardo et 
ah, 2002; Cano-Garcia & Hewitt Hughes, 2000; Ljell & Walhovd, 
2004; Sternberg & Grigorenko, 1993; Zhang & Sachs, 1997). 

The first goal of the study was met to the extent that we were able 
to specify the latent structure of the strongest items in the TSI and to 
identify and omit those items that did not meet standard psychomet- 


JEG 32(2) Winter 2008,indd 206 




12/12/08 8:55:57 AM 


207 





T5I Validity Study 




ric criteria. The second goal was partially met in the identification 
and retention of several original items whose structure was consistent 
with MSG theory that may serve as a basis for the creation of a mean- 
ingful and reliable tool for the measurement of thinking styles. 

The results of this study are limited to scores from a single sample 
of high school students from four high schools in Connecticut. In 
addition, only 12 of the 13 subscales were included in the analysis, 
and the psychometric properties of the Oligarchic subscale were 
not determined with this sample. The reported factor structures 
accounted for only 96 of the 104 items in the TSI. Nonetheless, the 
study contributes to a continually developing pattern of evidence 
highlighting the weaknesses in the current operationalization of 
MSG thinking styles. This collection of evidence brings into ques- 
tion the validity of the claims of previous research regarding the role 
of thinking styles in students’ academic experiences, and calls upon 
researchers and practitioners to be fully considerate of these psycho- 
metric weaknesses when making important educational decisions 
based upon such research. If educational practice is to be influenced 
by the notion of students’ thinking styles, we must first insist upon 
improved standards of measurement of that construct. 
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End Notes 

1 Data augmentation (DA) is an iterative data simulation tech- 
nique that involves the alternation of imputation of missing values 
and Bayesian parameter estimation steps until convergence is reached 
(Schafer, 2000). The option for single imputation at the end of the 
DA run, allowing for 1,000 iterations was selected in this case. 
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